On the Almost Everywhere Convergence of Nonparametric Regression Function Estimates by Luc Devroye

Let (X, Y), (X1, Y1), .•, (X,, Yn ) be independent identically distributed random vectors from R dxR, and let E(I Y 1 p) < oo for some p > 1 . We wish to estimate the regression function m(x) = E(Y I X = x) by mn(x), a function of x and (X1 , Y1), .•, (X,, Yn ) . For large classes of kernel estimates and nearest neighbor estimates, sufficient conditions are given for E { l m(x) m(x) 0 ) -* 0 as n -* oo, almost all x . No additional conditions are imposed on the distribution of (X, Y) . As a by-product, just assuming the boundedness of Y, the almost sure convergence to O of E { I m(X) m (X) I I Xl , Yl, • • • , X,, Yn } is established for the same estimates. Finally, the weak and strong Bayes risk consistency of the corresponding nonparametric discrimination rules is proved for all possible distributions of the data .